Reliable Computing with Unreliable Components: Using Separable Environments to Stabilize Long-Term Information Storage

How, in the face of both intrinsic and extrinsic volatility, can unconventional computing fabrics store information over arbitrarily long periods? Here, we argue that the predictable structure of many realistic environments, both natural and artificial, can be used to maintain useful categorical boundaries even when the computational fabric itself is inherently volatile and the inputs and outputs are partially stochastic. As a concrete example, we consider the storage of binary classifications in connectionist networks, although the underlying principles should be applicable to other unconventional computing paradigms. Specifically, we demonstrate that an unsupervised, activity dependent plasticity rule, AHAH (Anti-Hebbian-And-Hebbian), allows binary classifications to remain stable even when the underlying synaptic weights are subject to random noise. When embedded in environments composed of separable features, the weight vector is restricted by the AHAH rule to local attractors representing stable partitions of the input space, allowing unsupervised recovery of stored binary classifications following random perturbations that leave the system in the same basin of attraction. We conclude that the stability of long-term memories may depend not so much on the reliability of the underlying substrate but rather on the reproducible structure of the environment itself, suggesting a new paradigm for reliable computing with unreliable components.